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Abstract 

The formalism of statistical mechanics can be generalized by starting from more gen- 
eral measures of information than the Shannon entropy and maximizing those subject 
to suitable constraints. We discuss some of the most important examples of informa- 
tion measures that are useful for the description of complex systems. Examples treated 
are the Renyi entropy, Tsallis entropy, Abe entropy, Kaniadakis entropy, Sharma-Mittal 
entropies, and a few more. Important concepts such as the axiomatic foundations, com- 
posability and Lesche stability of information measures are briefly discussed. Poten- 
tial applications in physics include complex systems with long-range interactions and 
metastable states, scattering processes in particle physics, hydrodynamic turbulence, de- 
fect turbulence, optical lattices, and quite generally driven nonequilibrium systems with 
fluctuations of temperature. 



1 How to measure infor- 
mation 



1.1 Prologue 

How should one measure information? There is 
no unique answer to this. There are many dif- 
ferent information measures, and what measure 
of information is the most suitable one will in 
general depend on the problem under consider- 
ation. Also, there arc different types of infor- 
mation. For example, the information a reader 
gets from reading a book on quantum field the- 
ory is different from the one he gets from read- 
ing Shakespeare's Romeo and Juliet. In general 
one has to distinguish between elementary and 
advanced information concepts. The elementary 
information is just related to technical details 
such as, for example, the probability to observe 
certain letters in a long sequence of words. The 
advanced information is related to the informa- 
tion the reader really gets out of reading and 
understanding a given text, i.e. this concept re- 
quires coupling to a very complex system such 
as the brain of a human being. 

In physics, the missing information on the con- 
crete state of a system is related to the entropy of 
the system. Entropy is an elementary informa- 
tion concept. Many different physical definitions 
of entropy can be given, and what makes up a 
'physically relevant entropy' is often subject to 
'heated' discussions. Misunderstandings with re- 
spect to the name 'entropy' seem to be the rule 
rather than the exception within the past 130 
years. Generally one may use the name 'entropy' 
as a synonym for a possible quantity to measure 
missing information, keeping in mind that large 
classes of possible functions will potentially do 
the job, depending on application. 

The entire formalism of statistical mechanics 




Figure 1 : There is no obvious way to measure 
the information contents of given symbol se- 
quences. While it is relatively easy to distin- 
guish between a random sequence of symbols 
and Shakespeare's Romeo and Juliet in terms 
of suitable elementary information measures, 
it is less obvious how to distinguish the fact 
that the advanced information contents given 
by Shakespeare's Romeo and Juliet is differ- 
ent from the one given by a book on quantum 
field theory. 
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can be regarded as being based on maximizing 
the entropy (= missing information) of the sys- 
tem under consideration subject to suitable con- 
straints, and hence naturally the question arises 
how to measure this missing information in the 
first place [T]. While normally one chooses the 
Shannon information measure, in principle more 
general information measures (that contain the 
Shannon information SiS Si special case) can be 
chosen as well. These then formally lead to gen- 
eralized versions of statistical mechanics when 
they are maximized [H El d El H E] . 

In this paper we describe some generalized in- 
formation and entropy measures that are useful 
in this context. We discuss their most impor- 
tant properties, and point out potential physical 
applications. The physical examples we choose 
are the statistics of cosmic rays [8], defect tur- 
bulence p], and optical lattices [lOl E], but 
the general techniques developed have applica- 
tions for a variety of other complex systems as 
well, such as driven nonequilibrium systems with 
large-scale fluctuations of temperature (so-called 
superstatistical systems [El [13]), hydrodynamic 
turbulence [Ml [15] , scattering processes in par- 
ticle physics [HI [T7] , gravitationally interacting 
systems |181 [19] and Hamiltonian systems with 
long-range interactions and metastable states 
[201 121] . There are applications outside physics 
as well, for example in mathematical finance [22] . 
biology [23| and medicine [22]- 

1.2 Basic concepts 

One usually restricts the concept of an informa- 
tion measure to an information that is a func- 
tion of a given probability distribution of events 
(and nothing elsejll- The basic idea is as follows. 

"'^An exception to this rule is the Fisher informa- 
tion, which depends on gradients of the probability 



Consider a sample set of W possible events. In 
physics events are often identified as possible mi- 
crostates of the system. Let the probability that 
event i occurs be denoted as pi. One has from 
normalization 

w 

Ep^ = 1- (1) 

i=l 

We do not know which event will occur. But 
suppose that one of these events, say j, finally 
takes place. Then we have clearly gained some 
information, because before the event occurred 
we did not know which event would occur. 

Suppose that the probability pj of that ob- 
served event j is close to 1. This means we 
gain very little information by the observed oc- 
currence of event j, because this event was very 
likely anyway. On the other hand, if pj is close 
to zero, then we gain a lot of information by the 
actual occurrence of event j, because we did not 
really expect this event to happen. The informa- 
tion gain due to the occurrence of a single event 
j can be measured by a function h{pj), which 
should be close to zero for pj close to 1. For ex- 
ample, we could choose h{pj) = logpj, the loga- 
rithm to some suitable basis a. If this choice of a 
is a = 2 then h is sometimes called a 'bit-number' 
[1]. But various other functions h{pj) are possi- 
ble as well, depending on the application one has 
in mind. In other words, an information measure 
should better be regarded as a man-made con- 
struction useful for physicists who don't fully un- 
derstand a complex system but try to do so with 
their limited tools and ability. Once again we 
emphasize that an information measure is not a 
universally fixed quantity. This fact has led to 
many misunderstandings in the community. 

In a long sequence of independent trials, in 
order to determine an average information gain 

density but will not be discussed here. 
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by the sequence of observed events i we have to 
weight the information gain associated with a 
single event with the probabihty pi that event 
i actually occurs. That is to say, for a given 
function h the average information gained during 
a long sequence of trials is 

w 

I{{P^})=T.PMP^)■ (2) 
i=l 

Many information measures studied in the liter- 
ature are indeed of this simple trace form. But 
other forms are possible as well. One then de- 
fines the entropy S as 'missing information', i.e. 

S = -I. (3) 

This means the entropy is defined as our missing 
information on the actual occurrence of events, 
given that we only know the probability distri- 
bution of the events. 

If the probability distribution is sharply 
peaked around one almost certain event j, we 
gain very little information from our long-term 
experiment of independent trials: The event j 
will occur almost all of the time, which we al- 
ready knew before. However, if all events are 
uniformly distributed, i.e. pi = 1/W for all i, we 
get a large amount of information by doing this 
experiment, because before we did the experi- 
ment we had no idea which events would actu- 
ally occur, since they were all equally likely. In 
this sense, it is reasonable to assume that an (el- 
ementary) information measure should take on 
an extremum (maximum or minimum, depend- 
ing on sign) for the uniform distribution. More- 
over, events i that cannot occur (pi = 0) do not 
influence our gain of information in the experi- 
ment at all. In this way we arrive at the most 
basic principles an information measure should 
satisfy. 



1.3 The Khinchin axioms 

There is a more formal way to select suitable (el- 
ementary) information measures, by formulating 
a set of axioms and then searching for informa- 
tion measures that satisfy these axioms. A pri- 
ori, there is an infinite set of possible information 
measures, not only of the simple form ([2]) but of 
more general forms as well, based on arbitrary 
functions of the entire set of pi. How can we 
select the most suitable ones, given certain re- 
quirements we have in mind? Of course, what 
is 'most suitable' in this context will in general 
depend on the application we have in mind. The 
most appropriate way of dealing with this prob- 
lem is to postulate some basic and essential prop- 
erties the information measure one is interested 
in should have, and then to derive the functional 
form(s) that follows from these postulates. 

Khinchin [25] has formulated four axioms 
that describe the properties a 'classical' infor- 
mation measure / should have (by 'classical' we 
mean an information measure yielding ordinary 
Boltzmann-Gibbs type of statistical mechanics): 

Axiom 1 

I = l{pi,---,pw) (4) 
That is to say, the information measure I only 
depends on the probabilities pi of the events and 
nothing else. 

Axiom 2 

I{W-\...,W-^)<I{pi,---,Pw) (5) 

This means the information measure / takes on 
an absolute minimum for the uniform distribu- 
tion {W~^, . . . , W~^), any other probability dis- 
tribution has an information contents that is 
larger or equal to that of the uniform distribu- 
tion. 
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Axiom 3 

I{pi,...,pw) = I{pi,.--,Pw,0) (6) 

This means the information measure / should 
not change if the sample set of events is enlarged 
by another event that has probability zero. 

Axiom 4 

I{{plf'}) = I{{pi}) + T.Pi^({p''(3\i)}) (7) 

i 

This axiom is slightly more complicated and re- 
quires a longer explanation. The axiom deals 
with the composition of two systems I and II 
(not necessarily independent). The probabilities 
of the first system are pf, those of the second sys- 
tem are p^ . The joint system I, II is described by 
the joint probabilities p^j^^ = pjp^^ where 
p^^{j\i) is the conditional probability of event j 
in system II under the condition that event i has 
already occurred in system /. I{{p^^ is the 
conditional information of system II formed with 
the conditional probabilities p^^{j\i), i.e. under 
the condition that system I is in state i. 

The meaning of the above axiom is that it 
postulates that the information measure should 
be independent of the way the information is col- 
lected. We can first collect the information in the 
subsystem II, assuming a given event i in system 
I, and then sum the result over all possible events 
i in system I, weighting with the probabilities p^. 

For the special case that system I and II are 
independent the probability of the joint system 
factorizes as 

1,11 I II /o\ 

Pij =PiPj , (8) 

and only in this case, axiom 4 reduces to the 
rule of additivity of information for independent 
subsystems: 

I {{plf}) = I {{pi}) (9) 



Whereas there is no doubt about Axioms 1- 
3, the reader immediately notices that Axiom 
4 requires a much longer explanation. From a 
physical point of view, Axiom 4 is a much less 
obvious property. Why should information be 
independent from the way we collect it? 

To illustrate this point, we may consider a 
simple example of an information-collecting sys- 
tem, a first-year undergraduate student trying 
to understand physics. This student will learn 
much more if he first attends a course on classical 
mechanics, collecting all available information 
there, and then attends a course on quantum 
mechanics. If he does it the other way round, he 
will probably hardly understand anything in the 
course on quantum mechanics, since he does not 
have the necessary prerequisites. So attending 
the quantum mechanics course first leads to zero 
information gain. Apparently, the order in which 
the information of the two courses (the two sub- 
systems) is collected is very important and leads 
to different results in the achieved knowledge. 

In general complex systems, the order in which 
information is collected can be very relevant. 
This is a kind of information hysteresis phe- 
nomenon. In these cases we have situations 
where the replacement of Axiom 4 by something 
more general makes physical sense. We will come 
back to this in section 3. 

1.4 The Shannon entropy 

It is easy to verify that the celebrated Shannon 
entropy, defined by 

w 

S = -kY,Pilnpi (10) 

i=l 

satisfies all four of the Khinchin axioms. Indeed, 
up to an arbitrary multiplicative constant, one 
can easily show (see, e.g., [T]) that this is the 
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Figure 2: The grave of Boltzmann in Vi- 
enna. On top of the gravestone the formula 
S = klogW is engraved. Boltzmann laid 
the foundations for statistical mechanics, but 
his ideas were not widely accepted during his 
time. He comitted suicide in 1906. 

only entropic form that satisfies all four Khinchin 
axions, and that it follows uniquely (up to a mul- 
tiplicative constant) from these postulates, k de- 
notes the Boltzmann constant, which in the re- 
maining sections will be set equal to 1. For the 
uniform distribution, pi = 1/W, the Shannon 
entropy takes on its maximum value 



S = klnW, 



(11) 



which is Boltzmann's famous formula, carved on 
his grave in Vienna. Maximizing the Shannon 
entropy subject to suitable constraints leads to 
ordinary statistical mechanics (see section 4.2). 



In thermodynamic equilibrium, the Shannon en- 
tropy can be identified as the 'physical' entropy 
of the system, with the usual thermodynamic re- 
lations. Generally, the Shannon entropy has an 
enormous range of applications not only in equi- 
librium statistical mechanics but also in coding 
theory, computer science, etc. 

It is easy to verify that 5" is a concave func- 
tion of the probabilities pi, which is an important 
property to formulate statistical mechanics. Re- 
member that concavity of a differentiable func- 
tion f{x) means f"{x) < for all x. For the 
Shannon entropy one has 



dpi 

S 



dpidpj 



-Inpi - 1 



—5ij < 0, 
Pi 



(12) 
(13) 



and hence, as a sum of concave functions of the 
single probabilities pi, the Shannon entropy S is 
a concave function. 

In classical mechanics, one often has a con- 
tinuous variable u with some probability den- 
sity p{u), rather than discrete microstates i with 
probabilities pi. In this case the normalization 
condition reads J^^p{u)du = 1, and the Shan- 
non entropy associated with this probability den- 
sity is defined as 



S 



dup{u) ln((Tp(n)), 



(14) 



where o" is a scale parameter that has the same 
dimension as the variable u. For example, if u 
is a velocity (measured in units of m/s), then 
probability density of velocities, has 
the dimension s/m, since p{u)du is a dimension- 
less quantity. As a consequence, one needs to 
introduce the scale parameter a in eq. (I14p to 
make the argument of the logarithm dimension- 
less. 
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Besides the Shannon information, there are 
lots of other information measures. We will dis- 
cuss some of the most important examples in 
the next section. Some information measures are 
more suitable than others for the description of 
various types of complex systems. We will dis- 
cuss the axiomatic foundations that lead to cer- 
tain classes of information measures. Important 
properties to check for a given information mea- 
sure are convexity, additivity, composability, and 
stability. These properties can help to select the 
most suitable generalized information measure 
to describe a given class of complex systems. 

2 More general informa- 
tion measures 

2.1 The Renyi entropies 

We may replace axiom 4 by the less stringent 
condition ([9]), which just states that the entropy 
of independent systems should be additive. In 
this case one ends up with other information 
measures which are called the Renyi entropies. 
|26j . These are defined for an arbitrary real pa- 
rameter q as 

4''^ = -^lnE^^^ (15) 

The summation is over all events i with pi ^ 0. 
The Renyi entropies satisfy the Khinchin axioms 
1-3 and the additivity condition Indeed, 
they follow uniquely from these conditions, up 
to a multiplicative constant. For g — > 1 they 
reduce to the Shannon entropy: 

lim Sj.^'^ = S, (16) 

as can be easily derived by setting q = I + e 
and doing a perturbative expansion in the small 



parameter e in eq. (jlSp . 

The Renyi information measures are impor- 
tant for the characterization of multifractal sets 
(i.e., fractals with a probability measure on their 
support [I]), as well as for certain types of appli- 
cations in computer science. But do they provide 
a good information measure to develop a gener- 
alized statistical mechanics for complex systems? 

At first sight it looks nice that the Renyi en- 
tropies are additive for independent subsystems 
for general q, just as the Shannon entropy is for 
q = 1. But for non-independent subsystems 
I and II this simplicity vanishes immediately: 
There is no simple formula of expressing the to- 
tal Renyi entropy of a joint system as a simple 
function of the Renyi entropies of the interacting 
subsystems. 

Does it still make sense to generalize statistical 
mechanics using the Renyi entropies? Another 
problem arises if one checks whether the Renyi 
entropies are a convex function of the probabil- 
ities. The Renyi entropies do not possess a def- 
inite convexity — the second derivative with re- 
spect to the Pi can be positive or negative. For 
formulating a generalized statistical mechanics, 
this poses a serious problem. Other generalized 
information measures are better candidates-we 
will describe some of those in the following. 

2.2 The TsaUis entropies 

The TsaUis entropies (also called g-entropies) are 
given by the following expression ^: 

One finds definitions similar to eq. ()17p already 
in earlier papers such as e.g. [27], but it was Tsal- 
lis in his seminal paper [2] who for the first time 
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suggested to generalize statistical mechanics us- 
ing these entropic forms. Again g € 7^ is a real 
parameter, the entropic index. As the reader im- 
mediately sees, the Tsallis entropies are different 
from the Renyi entropies: There is no logarithm 
anymore. A relation between Renyi and Tsallis 
entropies is easily derived by writing 



$:p?=i-(,-i)5f) 



(18) 



which implies 



1 



(19) 



Apparently the Tsallis entropy is a monotonous 
function of the Renyi entropy, so any maximum 
of the Tsallis entropy will also be a maximum of 
the Renyi entropy and vice versa. But still, Tsal- 
lis entropies have many distinguished properties 
that make them a better candidate for general- 
izing statistical mechanics than, say, the Renyi 
entropies. 

One such property is concavity. One easily 
verifies that 



dpidpj 



q q~l 

7Pi 



-qPi ^ij- 



(20) 
(21) 



This means that, as a sum of concave functions, 

(T) 

Sq is concave for all g > (convex for all 
q < 0). This property is missing for the Renyi 
entropies. Another such property is the so-called 
Lesche-stability, which is satisfied for the Tsallis 
entropies but not satisfied by the Renyi entropies 
(see section 3.3 for more details). 

The Tsallis entropies also contain the Shannon 
entropy 



S 



w 

E 

i=l 



Pi In Pi 



(22) 



as a special case. Letting g — > 1 we have 



S^P = lim = S 



(23) 



As expected from a good information measure, 
the Tsallis entropies take on their extremum for 
the uniform distribution pi = 1/W Mi. This 
extremum is given by 



M/i-? _ 1 
l-q 



(24) 



which, in the limit ^ ^ 1, reproduces Boltz- 
mann's celebrated formula S = InW. 

It is also useful to write down the definition 
of Tsallis entropies for a continuous probability 
density p{u) with J^^p{u)du = 1, rather than 
a discrete set of probabilities pi with J2iPi = 1- 
In this case one defines 



q{t) 
jg 



1 



1 



1 



-oo 

-{apiuW], (25) 

X) o 



where again o" is a scale parameter that has the 
same dimension as the variable u. It is intro- 
duced for a similar reason as before, namely to 
make the integral in eq. ()25p dimensionless so 
that it can be substracted from 1. For g — > 1 
eq. (I25p reduces to the Shannon entropy 



SP = S= - r dup{u) \n{cjp{u)). (26) 

J —oo 

A fundamental property of the Tsallis en- 
tropies is the fact that they are not additive for 
independent subsystems. In fact, they have no 
chance to do so, since they are different from the 
Renyi entropies, the only solution to eq. ([9]). 

To investigate this in more detail, let us con- 
sider two independent subsystems I and II with 
probabilities pi and pY , respectively. The prob- 
abilities of joint events i,j for the combined sys- 
tem I,II are pij = pIpY ■ We may then consider 
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the Tsallis entropy for the first system, denoted 
as Sq, that of the second system, denoted as S^^ , 
and that of the joint system, denoted as S^'^^ . 
One has 



Proof of eq. ()27j) : We may write 
l-{q-l)Sl 

1-iq- 



(27) 



E4 



From eqs. ()28p and (j29p it also follows that 

EipiyEipyy = i-iq-i)si-{q-i)s'/ 

i 3 

+ {q-lfSlS'q'. (31) 




Combining eqs. (|30p and (13ip one ends up with 
eq. ([27]), q.e.d. 

Apparently, if we put together two indepen- 
dent subsystems then the Tsallis entropy is not 
additive but there is a correction term propor- 
tional to g — 1, which vanishes for q = 1 only, i.e. 
for the case where the Tsallis entropy reduces 
to the Shannon entropy. Eq. (|27p is sometimes 
called the 'pseudo-additivity' property. 

Eq. ()27p has given rise to the name nonexten- 
sive statistical mechanics. If we formulate a gen- 
eralized statistical mechanics based on maximiz- 
ing Tsallis entropies, then the (Tsallis) entropy 
of independent systems is not additive. How- 
ever, it turns out that for special types of cor- 
related subsystems, the Tsallis entropies do be- 
come additive if the subsystems are put together 
|28j . This means, for these types of correlated 
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Figure 3: If the nonadditive entropies Sg are 
used to measure information, then the infor- 
mation contents of two systems I, II (blue) 
that are put together is not equal to the sum 
of the information contents of the isolated 
single systems. In other words, there is al- 
ways an interaction between the subsystems 
(red). 
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complex systems a description in terms of Tsal- under the exchange q 
hs entropies in fact can make things simpler as 



q ^. This is given by 



compared to using the Shannon entropy, which 
is non- additive for correlated subsystems. 



S. 



Abe 



Pi 



q-q- 



(36) 



2.3 Landsberg-Vedral entropy 

Let us continue with a few other examples of 
generalized information measures. Consider 



1 



1 



1 



1 



(32) 



This symmetric choice in q and q~^ is inspired 
by the theory of quantum groups which often 
exhibits invariance under the 'duality transfor- 
mation' q — > q^^. Like Tsallis entropy, the Abe 
entropy is also concave. In fact, it is related to 
the Tsallis entropy ST by 



This measure was studied by Landsberg and 
Vedral [22]. One immediately sees that the 

Landsberg-Vedral entropy is related to the Tsal- 

(T) 

lis entropy Sq by 



S: 



Abe 



{q - 1)SJ - {q- 



q-q~ 



(37) 



Clearly the relevant range of q is now just the 
unit interval(0, 1], due to the symmetry q < — > 



q{l) 



AT) 



2^1=1 Pi 



q . Replacing q hy q in eq. ([36]) does not 
(33) change anything. 



and hence Sq^^ is sometimes also called normal- 2.5 Kaniadakis entropy 



ized Tsallis entropy. Sq^^ also contains the Shan- 
non entropy as a special case 

lim = Si (34) 

and one readily verifies that it also satisfies a 
pseudo-additivity condition for independent sys- 
tems, namely 

5f )^'^^ = 5f )^ + 5f + {q- l)5f )^5f 

(35) 

This means that in the pseudo-additivity rela- 
tion ([27|) the role of (g — 1) and — (g — 1) is ex- 
changed. 



2.4 Abe entropy 

Abe [30] introduced a kind of symmetric modifi- 
cation of the Tsallis entropy, which is invariant 



The Kaniadakis entropy (also called K-entropy) 
is defined by the following expression |1] 



5« = E 



1-ft 



2k 



(38) 



Again this is a kind of deformed Shannon en- 
tropy, which reduces to the original Shannon en- 
tropy for K = 0. We also note that for small 
K, and by writing q = 1 + k, q~^ ^ 1 — 
the Kaniadakis entropy approaches the Abe en- 
tropy. Kaniadakis was motivated to introduce 
this entropic form by special relativity: The rel- 
ativistic sum of two velocities of particles of mass 
m in special relativity satisfies a similar rela- 
tion as the Kaniadakis entropy does, identifying 
K = 1/mc. Kaniadakis entropies are also con- 
cave and Lesche stable (see section 3.3). 
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2.6 Sharma-Mittal entropies New Axiom 4 



These are two-parameter families of entropic 
forms [31j. They can be written in the form 



Pi - Pi 

2k 



(39) 



Interestingly, they contain many of the entropies 
mentioned so far as special cases. The Tsallis 
entropy is obtained for r = k and q = 1 — 2k. 
The Kaniadakis entropy is obtained for r = 0. 
The Abe entropy is obtained for k = ^{q — q~^) 
and r = ^{q + q^^) — 1. The Sharma-Mittal 
entropes are concave and Lesche stable. 



3 Selecting a suitable in- 
formation measure 

3.1 Axiomatic foundations 

The Khinchin axioms apparently are the right 
axioms to obtain the Shannon entropy in a 
unique way, but this concept may be too narrow- 
minded if one wants to describe general complex 
systems. In physics, for example, one may be 
interested in nonequilibrium systems with a sta- 
tionary state, glassy systems, long transient be- 
haviour in systems with long-range interactions, 
systems with multifractal phase space structure 
etc. In all these cases one should be open-minded 
to allow for generalizations of axiom 4, since it 
is this axiom that is least obvious in the given 
circumstances. 

Abe |32j has shown that the Tsallis entropy 
follows uniquely (up to an arbitrary multiplica- 
tive constant) from the following generalized ver- 
sion of the Khinchin axioms. Axioms 1~3 are 
kept, and Axiom 4 is replaced by the following 
more general version: 



S'/' = S'g + 5f 1^ -{q- 1)SIS!,'\' (40) 

Here Sq^^^ is the conditional entropy formed with 
the conditional probabilities p{j\i) and averaged 
over all states i using the so-called escort distri- 
butions P,-: 



5fl^ = ^P.5,(Mj!.)}). 



(41) 



Escort distributions Pi were introduced quite 
generally in [T] and are defined for any given 
probability distribution p, by 



Pi 



T,iP 



I Pi 



(42) 



For q = 1, the new axiom 4 reduces to the old 
Khinchin axiom 4, i.e. S^'^^ = + Sq^^^ . For 
independent systems I and II, the new axiom 4 
reduces to the pseudo-additivity property ([27|) . 

The meaning of the new axiom 4 is quite clear. 
It is a kind of minimal extension of the old ax- 
iom 4: If we collect information from two subsys- 
tems, the total information should be the sum of 
the information collected from system I and the 
conditional information from system II, plus a 
correction term. This correction term can a pri- 
ori be anything, but we want to restrict ourselves 
to information measures where 



(43) 



where g{x, y) is some function. The property 
that the entropy of the composed system can be 
expressed function of the entropies of the 
single systems is sometimes referred to as the 
composability property. Clearly, the function g 
must depend on the entropies of both subsystems, 
for symmetry reasons. The simplest form one 
can imagine is that it is given by 



g{x, y) = const • xy, 



(44) 
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i.e. it is proportional to both the entropy of the 
first system and that of the second system. Call- 
ing the proportionality constant q — 1, we end up 
with the new axiom 4. 

It should, however, be noted that we may well 
formulate other axioms, which then lead to other 
types of information measures. The above gen- 
eralization is perhaps the one that requires least 
modifications as compared to the Shannon en- 
tropy case. But clearly, depending on the class 
of complex systems considered, and depending 
on what properties we want to describe, other 
axioms may turn out to be more useful. For ex- 
ample, Wada and Suyari [33] have suggested a 
set of axioms that uniquely lead to the Sharma- 
Mittal entropy. 

3.2 Composability 

Suppose we have a given complex system which 
consists of subsystems that interact in a compli- 
cated way. Let us first analyze two subsystems 
I and II in an isolated way and then put these 
two dependent systems I and II together. Can we 
then express the generalized information we have 
on the total system as a simple function of the 
information we have on the single systems? This 
question is sometimes referred to as the compos- 
ability problem. 

The Tsallis entropies are composable in a very 
simple way. Suppose the two systems I and II are 
not independent. In this case one can still write 
the joint probability Pij Q-S Si product of the single 
probability pi and conditional probability p{j\i), 
i.e. the probability of event j under the condition 
that event i has already occurred is 

Pij =p{i\j)Pi- (45) 

The conditional Tsallis entropy associated with 
system II (under the condition that system I is 



in state i) is given by 

^f'^ = ^(i-E^^(jm- (46) 

One readily verifies the relation 

Si + J2PK\' = S'/'. (47) 

i 

This equation is very similar to that satisfied by 
the Shannon entropy in Axiom 4. In fact, the 
only difference is that there is now an exponent q 
that wasn't there before. It means our collection 
of information is biased: Instead of weighting 
the events in system I with pi we weight them 
with pf. For q = 1 the above equation of course 
reduces to the fourth of the Khinchin axioms, 
but only in this case. Hence, for general q ^ 1, 
the Tsallis information is not independent of the 
way it is collected for the various subsystems. 

To appreciate the simple composability prop- 
erty of the Tsallis entropy, let us compare with 
other entropy- like functions, for example the 
Renyi entropy. For the Renyi entropy there 
is no simple composability property similar to 
eq. ()47p . Only the exponential of the Renyi 
entropy satisfies a relatively simple equation, 
namely 

exp ((, - l)Sf )^'^^) = Ep^xp ((, - l)5f . 

(48) 

However, by taking the exponential one clearly 
removes the logarithm in the definition of the 
Renyi entropies in eq. (jl5p . This means one is 
effectively back to the Tsallis entropies. 

3.3 Lesche stability 

Physical systems contain noise. A necessary re- 
quirement for a generalized entropic form S[p] to 
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make physical sense is that it must be stable un- 
der small perturbations. This means a small per- 
turbation of the set of probabilities p := {pi} to a 
new set p' = {p'^} should have only a small effect 
on the value Smax of Sq [p] in the thermodynamic 
state that maximizes the entropy. This should 
in particular be true in the limit W ^ oo (re- 
call that W denotes the number of microstates) . 
The stability condition can be mathematically 
expressed as follows 



Stability condition 

For every e > there is a 5 > such that 



\\p-p'\\i < S 



S[p] - S[p'] 



< e (49) 



\Ai 



for arbitrarily large W. Here = J2l= 

denotes the Li norm of an observable A. 

Abe [35] has proved that the Tsallis entropies 
are Lesche-stable, i.e. they satisfy eq. (j49]l for all 
q, whereas the Renyi entropies and the Lands- 
berg entropies are not stable for any q ^ 1 (for 
a discrete set of probabilities pi with — s- oo). 
This is an important criterion to single out gen- 
eralized entropies that may have potential phys- 
ical meaning. According to the stability crite- 
rion, the Tsallis entropies are stable and thus 
may be associated with physical states, whereas 
the other two examples of entropic forms have 
a stability problem. Kaniadakis entropies and 
Sharma Mittal entropies are also Lesche-stable. 
Only entropies that are Lesche-stable are good 
candidates for physically relevant information 
measures. For a recent re- investigation of these 
types of stability problems in a physical setting, 
see [361. 



4 Maximizing generalized 
entropies 

4.1 A rigorous derivation of 
statistical mechanics? 

The rigorous foundations of statistical mechan- 
ics are a kind of miracle. There is little progress 
in rigorously deriving statistical mechanics from 
the microscopic classical Hamiltonian equations 
of motion, neither there is a rigorous derivation 
starting from quantum mechanics or quantum 
field theory. It is almost surprising how well 
statistical mechanics works in daily life, given 
its lack of rigorous derivation from other micro- 
scopic theories. 

The problem is that for a rigorous deriva- 
tion of statistical mechanics from dynamical sys- 
tems theory one needs the underlying dynami- 
cal system to be ergodic, and even that is not 
enough: It should have the stronger property of 
mixing. Ergodicity essentially means that typ- 
ical trajectories fill out the entire phase space 
(which implies that for typical trajectories the 
time average is equal to the ensemble average) 
and mixing means asymptotic independence, i.e. 
the correlation function of distant events decays 
to zero if the time difference between the events 
goes to infinity. For strongly chaotic dynamical 
systems (i.e. those exhibiting exponential sensi- 
tive dependence on the initial conditions) one 
normally expects the mixing property to hold 
(though there are some mathematical subtleties 
here). From a mathematical point of view, the 
mixing property is the theoretical ingredient that 
is needed to guarantee the approach to an equi- 
librium state in statistical mechanics. 

Unfortunately, ergodicity and mixing can only 
be rigorously proved for simple toy examples 
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of dynamical systems, for example the discrete- 
time map Xn+i = 1 — 2x^ with initial values in 
the interval [—1,1] or other very simple toy mod- 
els (see, e.g. [1]). For realistic systems of physi- 
cal relevance, such as the Hamiltonian equations 
of a large number of weakly or strongly inter- 
acting particles, a rigorous mathematical proof 
of the mixing property does not exist, and the 
deeper reason why statistical mechanics works 
so well in typical situations remains a miracle. 

4.2 Jaynes' information theory 

In view of the fact that there are no rigorous 
foundations of statistical mechanics, one usually 
sticks to some simple principle such as the max- 
imum entropy principle in order to 'derive' it. 
Jaynes has given a simple and plausible inter- 
pretation of the maximum entropy principle [37] . 
His interpretation is purely based on concepts 
from information theory, and applicable to many 
problems, not only to equilibrium statistical me- 
chanics. 

In simple words, the idea is as follows. Assume 
we have only limited information on a system 
containing many particles or constituents. We 
may know the mean values of some observables 
M°', a = 1, • • • , s but nothing else. For example, 
we may just know one such quantity, the mean 
energy of all particles and nothing else {s = 1). 
What probability distributions pi should we now 
assume, given that we have such limited informa- 
tion on the system? 

Suppose we measure information with some 
information measure I{{pi}) =: I[p]. Among 
all distributions possible that lead to the above 
known mean values M'^ we should select those 
that do not contain any unjustified prejudices. 
In other words, our information measure for the 
relevant probabilities should take on a minimum. 



or the entropy (= negative information) should 
take a maximum, given the constraints. For, 
if the information associated with the selected 
probability distribution does not take on a min- 
imum, we have more information than the min- 
imum information, but this means we are pre- 
occupied by a certain belief or additional infor- 
mation, which we should have entered as a con- 
dition of constraint in the first place. 

Of course, if we have no knowledge on the 
system at all (s = 0), the principle yields the 
uniform distribution pi = 1/W, i = 1, . . .W of 
events. For this to happen, the information mea- 
sure I\p] must only satisfy the second Khinchin 
axiom, nothing else. In statistical mechanics, 
the corresponding ensemble is the microcanon- 
ical ensemble. 

If some constraints are given, we have to min- 
imize the information (= maximize the entropy) 
subject to the given constraints. A constraint 
means that we know that some observable M of 
the system, which takes on the values Mj in the 
microstates i, takes on the fixed mean value M. 
In total, there can be s such constraints, corre- 
sponding to s different observables M"^: 

J2piM[ = M'' {a = l,...,s). (50) 

i 

For example, for the canonical ensemble of equi- 
librium statistical mechanics one has the con- 
straint that the mean value U of the energies Ei 
in the various microstates is fixed: 

Y.PiE, = U (51) 

i 

We may also regard the fact that the probabili- 
ties Pi are always normalized as a constraint ob- 
tained for the special choice M = 1: 

Y.Pi = ^- (52) 

i 
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To find the distributions that maximize the 
entropy under the given constraints one can use 
the method of Lagrange multipliers. One simply 
defines a function ^[p] which is the information 
measure under consideration plus the condition 
of constraints multiplied by some constants (3o- 
(the Lagrange multipliers): 

^b]=^W+E/3-(EKMf). (53) 

o" i 

One then looks for the minimum of this func- 
tion in the space of all possible probabilities pi. 
In practice, these distributions pi are easily ob- 
tained by evaluating the condition 

^^[p] = {i = l,...,W), (54) 

which means that ^ has a local extremum. We 
obtain 

^J[p]+J2(3,M[ = (55) 

All this is true for any information measure 
it need not be the Shannon information. At this 
point we see why it is important that the infor- 
mation measure I\p] is convex: We need a well- 
defined inverse function of -^I[p], in order to 
uniquely solve eq. ()55p for the pi. This means 
-^I[p] should be a monotonous function, which 
means that I\p] must be convex. 

Note that Jaynes' principle is (in principle) 
applicable to all kinds of complex systems, many 
different types of observables, and various types 
of information measures. There is no reason 
to restrict it to equilibrium statistical mechan- 
ics only. It's generally applicable to all kinds 
of problems where one has missing information 
on the actual microscopic state of the system 
and wants to make a good (unbiased) guess of 
what is happening and what should be done. 
The concept of avoiding unjustified prejudices 



applies in quite a general way. An important 
question is which information measure is relevant 
for which system. Clearly, the Shannon entropy 
is the right information measure to analyse stan- 
dard type of systems in equilibrium statistical 
mechanics. But other systems of more complex 
nature can potentially be described more effec- 
tively if one uses different information measures, 
for examples those introduced in the previous 
section. 

4.3 Ordinary statistical me- 
chanics 

For ordinary statistical mechanics, one has 
Ap] = Y^iPi^^Pi and -^J[p] = l + \n Pi. For the 
example of a canonical ensemble eq. (j53|) reads 

i it 

and eq. (j55]) leads to 

Inpi + l + a + (3Ei = 0. (57) 

Hence the maximum entropy principle leads to 
the canonical distributions 

Pi = ^e-'5^^ (58) 

The partition function Z is related to the La- 
grange multiplier a by 

Z:=^e-^^' = e^+". (59) 

i 

4.4 Generalized statistical me- 
chanics 

More generally we may start from a generalized 
information measure of the trace form 

m=-S[p]=T.P^HP^) (60) 
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where h is some suitable function, as intro- 
duced before. Tsallis entropy, Abe entropy, 
Kaniadakis entropy, Sharma-Mittal entropy and 
Shannon entropy are examples that can all be 
cast into this general form, with different func- 
tions h of course. Again let us consider the 
canonical ensemble (the extension to further 
constraints/other ensembles is straightforward). 
The functional to be maximized is then 

^\p]=Y.PMPi)+OcY.Pi + PY.Pi^i (61) 
i i i 

By evaluating the condition 

= (62) 

we obtain 

h{pi) + Pih'ipi) + a + pEi = {) (63) 

Defining a function g by 

g{Pi) •■= h{pi) +Pih'{pi) (64) 

we end up with 

gip^) = -a- pEi (65) 

Hence, if a unique inverse function g~^ exists, we 
have 

Pi = g-\-a - m) (66) 

and this is the generalized canonical distribution 
of the generalized statistical mechanics. 

Let us consider a few examples of interesting 
functions functions h. For the Shannon entropy 
one has of course 

h{pi) = Inpi. (67) 
For the Tsallis entropy, 

h{Pi) = ^^^^ =: log2-,(Pi). (68) 



This is like a deformed logarithm that ap- 
proaches the ordinary logarithm for q ^ I. In 
fact, a useful definition commonly used in the 
field is the so-called g-logarithm defined by 

— 1 

loggix) := . (69) 

Its inverse function is the g-exponential 

:= (l + (l-g)x)T^. (70) 
For the Kaniadakis entropy one has 

hip^) = =■■ ln«(x), (71) 

where the K-logarithm is defined as 

ln.(x) = (72) 
Its inverse is the ^-exponential 

exp^ix) = {^1 + hP'x'^ + Kx)'^ (73) 

Essentially, the generalized canonical distribu- 
tions obtained by maximizing Tsallis entropies 
are given by q- exponentials of the energy Ei and 
those by maximizing Kaniadakis entropies are k- 
exponentials. Both decay with a power law for 
large values of the energy E^. 

4.5 Nonextensive statistical 
mechanics 

Let us consider in somewhat more detail a gener- 
alized statistical mechanics based on Tsallis en- 
tropies. If we start from the Tsallis entropies 

(T) 

Sq and maximize those subject to suitable con- 
straint, the corresponding formalism is called 
nonextensive statistical mechanics. We have 
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thus 



q q-l 



V 



For a canonical ensemble eq. 



leads to 



q-l 



(75) 



(76) 



Thus the maximum entropy principle leads to 
generalized canonical distributions of the form 



Pi 



9-1 , 



(77) 



where Zq is a normalization constant. This is 
the original formula Tsallis introduced in his pa- 
per [2]. These days, however, the convention has 
become to replace the parameter q hy q' = 2 — q 
and then rename q' — > q. That is to say, the gen- 
eralized canonical distributions in nonextensive 
statistical mechanics are given by the following 
(/-exponentials: 



Pi 



m) 



q-l , 



(78) 



They live on a bounded support for q < 1 and 
exhibit power-law decays for q > 1. 

Starting from such a g-generalized approach, 
one can easily derive formal (/-generalized ther- 
modynamic relations. The details depend a bit 
how the constraints on energy taken into account 
[3]. All relevant thermodynamic quantities now 
get an index q. Typical examples of such formu- 
las are 



1/T = /? = a^f V^f^g, vg 



with 



W 



Fq = Uq — TSq 



1 (ZqY-'^ - 1 



i=l 



13 1 



(79) 

(80) 
(81) 



and 



where 



dp l-q 



{Zq)^-'^ - 1 _ [ZqY-'i - 1 

1 — q 1 — q 



(82) 



PUq. (83) 



and 



w 

i=l 



l^i=l Pi Ej 



(84) 



q 

2^i=l Pi 

There are some ambiguities on how to take into 
account the constraints, using for example the 
original pi or the escort distributions Pi, but we 
will not comment on these technicalities here. 

5 Some physical examples 

5.1 Making contact with exper- 
imental data 

It should be clear that a direct physical mea- 
surements of generalized entropy measures is 
impossible since these are basically man-made 
information-theoretic tools. However, what can 
be measured is the stationary probability distri- 
bution of certain observables of a given complex 
system, as well as possibly some correlations be- 
tween subsystems. As we have illustrated before, 
measured probability densities in some complex 
system that deviate from the usual Boltzmann 
factor e~^^ can then be formally interpreted as 
being due to the maximization of a more gen- 
eral information measure that is suitable as an 
effective description for the system under consid- 
eration. 

In this approach one regards the complex sys- 
tem as a kind of 'black box'. Indeed many phe- 
nomena in physics, biology, economics, social sci- 
ences, etc. are so complicated that there is not a 
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simple equation describing them, or at least we 
do not know this equation. A priori we do not 
know what is the most suitable way to measure 
information for any output that we get from our 
black box. But if a distribution pi of some ob- 
servable output is experimentally measured, we 
can indirectly construct a generalized entropic 
form that takes a maximum for this particular 
observed distribution. This allows us to make 
contact with experimental measurements, make 
some predictions e.g. on correlations of subsys- 
tems and translate the rather abstract informa- 
tion theoretical concepts into physical reality. 

5.2 Statistics of cosmic rays 

Our first example of making contact to concrete 
measurements is cosmic ray statistics [8j. The 
earth is constantly bombarded with highly ener- 
getic particles, cosmic rays. Experimental data 
of the measured cosmic ray energy spectrum are 
shown in Fig. 4. It has been known for a long 
time that the observed distribution of cosmic 
rays with a given energy E exhibits strongly pro- 
nounced power laws rather than exponential de- 
cay. It turns out that the observed distribution 
is very well fitted over a very large range of en- 
ergies by the formula 



p{E) = C 



(85) 



(1 + 6(g- 1)^)1/(9-1)- 
Here E is the energy of the cosmic ray particles, 

(86) 



E 



b = {kT)~^ is an effective inverse temperature 
variable, and C is a constant representing the 
total flux rate. For highly relativistic particles 
the rest mass m can be neglected and one has 
E ~ c\p\. The reader immediately recognizes 
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Figure 4: Measured energy spectrum of cos- 
mic rays and a fit by eq. fl85l) with q = 1.215. 
The 'knee' and 'ankle' are structures that go 
beyond the simple model considered here. 

the distribution ([85]) as a q-generalized relativis- 
tic Maxwell-Boltzmann distribution, which max- 
imizes the Tsallis entropy. The factor E'^ takes 
into account the available phase space volume. 
As seen in Fig. 4, the cosmic ray spectrum is 
very well fitted by the distribution (j85p if the 
entropic index q is chosen as q = 1.215 and if 
the effective temperature parameter is given by 
kT = = 107 MeV. Hence the measured cos- 
mic ray spectrum effectively maximizes the Tsal- 
lis entropy. 

The deeper reason why this is so could be tem- 
perature fluctuations during the production pro- 
cess of the primary cosmic ray particles [8] . Con- 
sider quite generally a superposition of ordinary 
Maxwell-Boltzmann distributions with different 
inverse temperatures /?: 



p{E) ~ / /(/3)i^^e-'3^d/3 



(87) 
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Here is the probability density to observe a 
given inverse temperature (3. If f{f3) is a Gamma 
distribution, then the integration in eq. (|87p can 
be performed and one ends up with eq. ([85]) (see 
[8] for more details). This is the basic idea under- 
lying so-called superstatistical models [12]: One 
does a kind of generalized statistical mechanics 
where the inverse temperature /? is a random 
variable as well. 

The effective temperature parameter T (a 
kind of average temperature in the above super- 
statistical model) is of the same order of magni- 
tude as the so-called Hagedorn temperature Th 
[38j . an effective temperature well known from 
collider experiments. The fact that we get from 
the fits something of the order of the Hagedorn 
temperature is encouraging. The Hagedorn tem- 
perature is much smaller than the center-of-mass 
energy Ecms of a typical collision process and 
represents a kind of 'boiling temperature' of nu- 
clear matter at the confinement phase transition. 
It is a kind of maximum temperature that can be 
reached in a collision experiment. Even largest 
Ecms cannot produce a larger average temper- 
ature than Th due to the fact that the number 
of possible particle states grows exponentially. 

Similar predictions derived from nonextensive 
statistical mechanics also fit measured differen- 
tial cross sections in e~^e~ annihilation processes 
and other scattering data very well (see e.g. 
[161 [TT] for more details). The hadronic cas- 
cade process underlying these scattering data is 
not well understood, though it can be simulated 
by Monte Carlo simulations. If we don't have 
any better theory, then the simplest model to 
reproduce the measured cross sections is indeed 
a generalized Hagedorn theory where the Shan- 
non entropy is replaced by Tsallis entropy [TTj. 



5.3 Defect turbulence 

Our next example is so-called 'defect turbu- 
lence'. Defect turbulence shares with ordinary 
turbulence only the name as otherwise it is very 
different. It is a phenomenon related to con- 
vection and has nothing to do with fully de- 
veloped hydrodynamic turbulence. Consider a 
Rayleigh-Benard convection experiment: A liq- 
uid is heated from below and cooled from above. 
For large enough temperature differences, inter- 
esting convection patterns start to evolve. An 
inclined layer convection experiment [9] is a kind 
of Rayleigh-Benard experiment where the appa- 
ratus is tilted by an angle (say 30 degrees) , more- 
over the liquid is confined between two very nar- 
row plates. For large temperature differences, 
the convection rolls evolve chaotically. Of par- 
ticular interest are the defects in this pattern, 
i.e. points where two convection rolls merge into 
one (see Fig. 5). These defects behave very much 
like particles. They have a well-defined position 
and velocity, they are created and annihilated 
in pairs, and one can even formally attribute a 
'charge' to them: There are positive and nega- 
tive defects, as indicated by the black and white 
boxes in Fig. 5. But the theory underlying these 
highly nonlinear excitations is pretty unclear, 
they are like a 'black box' complex system whose 
measured output is velocity. 

The probability density of defect velocities has 
been experimentally measured with high statis- 
tics [9]. As shown in Fig. 6, the measured dis- 
tribution is well fitted by a g-Gaussian with 
q ~ 1.45. The defects are also observed to ex- 
hibit anomalous diffusion. Their position X{t) 
roughly obeys an anomalous diffusion law of the 
type 

{X\t))r.r, (88) 

with a ~ 1.3. The relation a ~ 2/(3 — q) can be 
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Figure 5: Convection rolls and defects (black 
and white boxes) as observed in the experi- 
ment of Daniels et al. [H] 
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theoretically derived [9]. 

Apparently defects are a very complicated 
nonlinear system with complicated interactions 
in a nonequilibrium environment. Their dynam- 
ics is not fully understood so far. But we see 
that effectively they seem to behave like a gas 
of nonextensive statistical mechanics that leads 
to g-exponential Boltzmann factors rather than 
ordinary Boltzmann factors. 



5.4 Optical lattices 



Optical lattices are standing-wave potentials ob- 
tained by superpositions of counter-propagating 
laser beams. One obtains easily tunable periodic 
potentials in which atoms can perform normal 
and anomalous quantum transport processes. If 
the potential is very deep, there is diffusive mo- 
tion. If it is very shallow, there is ballistic mo- 
tion. In between, there is a regime with anoma- 
lous diffusion that is of interest here. 

Optical lattices can be theoretically described 
by a nonlinear Fokker-Planck equation for the 
Wigner function W{p,t) (the Wigner function 
is an important statistical tool for the quan- 
tum mechanical description in the phase space). 
The above Fokker-Planck equation admits Tsal- 
lis statistics as a stationary solution. This was 
pointed out by Lutz [lOj. The equation is given 

by 
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Figure 6: Measured probability density of de- is a momentum-dependent drift force and 
feet velocities and fit with a g-Gaussian with 
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a momentum-dependent diffusion constant. The 
stationary solution is 



W{p) = C- 



1 



{1 + f3{q - 1)E) — 



where 



E 



-p 



a 



Do + Di 
2Do 



1 + 



(92) 

(93) 
(94) 

(95) 



So the optical lattice effectively maximizes Tsal- 
lis entropy in its nonequilibrim stationary state. 
Another way to express the entropic index in 
terms of physical parameters is the formula 



1 + 



44^ 
Uo 



(96) 



where Eji is the so-called recoil energy and 
Uq the potential depth. These types of q- 
exponential predictions have been experimen- 
tally confirmed [TT]. Lutz' microscopic theory 
thus yields a theory of the relevant entropic in- 
dex q in terms of system parameters. 

5.5 Epilogue 

There are many other examples of physical sys- 
tems where generalized entropies yield a useful 
tool to effectively describe the complex system 
under consideration. Important examples in- 
clude Hamiltonian systems with long-range in- 
teractions that exhibit metastable states |20ll21j 
as well as driven nonequilibrium systems with 
large-scale fluctuations of temperature or en- 
ergy dissipation, i.e. superstatistical systems 
[T2I [39| [40] . The best way to define generalized 



entropies for superstatistical systems is still sub- 
ject of current research [6ll3Tl[l2]. Superstatisti- 
cal turbulence models yield excellent agreement 
with experimental data [13\ [T^ [T5] . Generalized 
statistical mechanics methods have also applica- 
tions outside physics, for example in mathemat- 
ical finance [22\ I43j . for traffic delay statistics 
|44j or in the medical [M] and biological sciences 
[7J. It is often in these types of complex sys- 
tems that one does not have a concrete equa- 
tion of motion and hence is forced to do cer- 
tain 'unbiased guesses' on the behaviour of the 
system — which for sufficiently complex systems 
may lead to other entropic forms than the usual 
Shannon entropy that are effectively maximized. 
The beauty of the formalism is that it can be ap- 
plied to a large variety of complex systems from 
different subject areas, without knowing the de- 
tails of the dynamics. 
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